Bootstrapping Hypothesis Testings in Credit Risk

Bootstrapping in Credit Risk

• Generally underutilized in the credit risk area. Particularly useful for testing metrics where there is little consensus on standard error or when statistical testing procedures are absent. Very often, even the inverted p-values of confidence intervals are sufficiently informative.

• Advantages:

  • does not rely on any distributional assumption
  • flexible application across a wide range of metrics
  • useful for identifying bias
  • easy to implement.

• Disadvantages:

  • computational intensity
  • assumption of independence (although this can be addressed to a certain extent)
  • accuracy concerns for smaller sample sizes.

Example 1: Population Stability Index (1-sided test)

Testing Hypothesis:
What is the probability that PSI value is less or equal to 0.15?
Visualization:

## p-value = 21.26%

Example 2: Herfindahl-Hirschman Index (1-sided test)

Dataset:

##         Rating Grade # obs.   DR
## 1   01 (-Inf,0.0199)    202 0.01
## 2 02 [0.0199,0.0263)     54 0.02
## 3 03 [0.0263,0.0369)     96 0.03
## 4 04 [0.0369,0.0903)    204 0.06
## 5   05 [0.0903,0.15)    103 0.11
## 6    06 [0.15,0.197)     41 0.12
## 7     07 [0.197,Inf)     50 0.32

Testing Hypothesis:
What is the probability that HHI value is greater or equal to 0.20?

Visualization:

## p-value = 23.72%

Example 3: Area Under Curve (2-sided test)

Dataset:

## Bootstrapped AUC summary:

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.6769  0.7414  0.7526  0.7525  0.7638  0.8067

## Development sample AUC 79%.

## Application portfolio AUC 75.2%.

Testing Hypothesis:
What is the probability that the application portfolio AUC is equal to 79%?

Visualization:

## 2*min(c(left-side p-value, right-side p-value))

## p-value = 2.06%